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Abstract. In this paper we prove large deviations principles for the av- 
>J > , eraged stochastic approximation method for the estimation of a regression 

function introduced by A. Mokkadem et al. [Revisiting Revesz's stochastic 
approximation method for the estimation of a regression function, ALEA 
Lat. Amm. J. Probab. Math. Stat. 6 (2009), 63-114]. We show that the 
averaged stochastic approximation algorithm constructed using the weight 
sequence which minimize the asymptotic variance gives the same point- 
(^ ' wise LDP as the Nadaraya- Watson kernel estimator. Moreover, we give 

a moderate deviations principle for these estimators. It turns out that 
the rate function obtained in the moderate deviations principle for the av- 
eraged stochastic approximation algorithm constructed using the weight 
sequence which minimize the asymptotic variance is larger than the one 
obtained for the Nadaraya- Watson estimator and the one obtained for the 
semi-recursive estimator. 
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1. Introduction 



Let {X, Y) , {Xi,Yi) ,... , {Xn, Yn) be independent, identically distributed pairs 
of random variables with joint density function g(x,y), and let / denote the 
^^ ■ probability density oi X. In order to construct a stochastic algorithm for the 

estimation of the regression function r : x i-^ ¥, {Y\X = x) a.t a point x such that 
/(x) ^ 0, A. Mokkadem et al. [5] defines an algorithm, which approximates 
the zero of the function h : y i-^ f{x)r{x) — f{x)y. Following Robbins-Monro's 
procedure, this algorithm is defined by setting ro(a;) G K and, for n > 1, 



r„(x) = r„_i(x) + 7„yV„(a;) 

where yV„(a;) is an "observation" of the function h at the point r„_i(a;). To 
define >V„(a;), A. Mokkadem et al [9] follow the approach of P. Revesz ([H], [l2] ) 
and A. B. Tsybakov [13], and introduces a kernel K (that is, a function satisfying 
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J^K(x)dx = 1) and a bandwidth (/i„) (that is, a sequence of positive real 
numbers that goes to zero), and sets 

W„(x) = h-'YnK{h-^[x - Xn]) - K^K{h~^[x ~ X„])r„_i(x). 

Then, the estimator r^ can be rewritten as 



— Ir^/"^ ^\\ /\, 1 —l^r T^ ( "^ '^ 



rn{x) = \l~lnh-'K \—^jj rn-l{x)+^nh-'YnK [-^^ j ■ (1-1) 

Now, let the stepsize in (jl.ip satisfy lim„_>oo f^7n — oo, and let (g„) be a pos- 
itive sequence such that X)?" — o°- The averaged stochastic approximation 
algorithm for the estimation of a regression function is defined by setting 



1 " 

rn{x) = ^:^, '^qkTkix) (1.2) 



(where the rk{x) are given by the algorithm (|l.ip V 

Recently, large and moderate deviations results have been proved for the 
well-known nonrecursive Nadaraya- Watson's kernel regression estimator, first 
by Louani (1999), and then by C. Joutard \4\. A. Mokkadem et al [S] show that 
the rate function obtained in the moderate deviations principle for the semi- 
recursive estimator is larger than the one obtained for the Nadaraya- Watson 
estimator. 

Let us first recall that a R'"-valued sequence (^n)„>i satisfies a large devia- 
tions principle (LDP) with speed (vn) and good rate function / if : 

(1) (t^„) is a positive sequence such that lim„_j.oo t^n = oo; 

(2) / : M™ — > [0, oo] has compact level sets; 

(3) for every borel set S C M™, 

-- inf / {x) < hm inf z^^^ log P [Z„ E B] 

o n— >oo 

xeB 

< lim sup V- ^ log P [Zn eB]<- in£ I (x) , 

n—>QO x^B 

o 

where B and B denote the interior and the closure of B respectively. 

Moreover, let (u„) be a nonrandom sequence that goes to infinity; if 

(vnZn) satisfies a LDP, then (Z„) is said to satisfy a moderate deviations 

principle (MDP). 
The first aim of this paper is to establish pointwise LDP for the averaged 
stochastic approximation algorithm (jl.2p . It turns out that the rate function 
depend on the bandwidths (/i„) and on the weight ((?«)■ 

We show that using the bandwidths (ft.„) = {cn^'^) with c > and a G 
]1 - a, (4a - 3) /2[ (with a e]f , 1]), and the weight (q„) = (c'n"') with c' > 
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and q < min{l — 2a, (1 + a) /2}, the sequence (r„ (x) — r (x)) satisfies a LDP 
with speed (ri/i„) and the rate function defined as follows: 



la.q.x (t) = sup {ut - Ipa.q.x (")} , 

uGR 

which is the Fenchel-Legendre transform of the function ^pa,q,x defined as follows: 
^a,q,x (u) = {l-q) I s'- (e"^"-'^(-)^^7f^ _ i) g (^, y) dsdzdy. (1.3) 

J[0,l]xR2 ^ ^ 

Noting that, in the special case (qn) = {h,i), which is the case when the weight 
(qn) minimizes the asymptotic variance of f„ (see A. Mokkadem et al., [9 ), we 
obtain the same rate function for the pointwise LDP as the one obtained for the 
Nadaraya-Watson estimator (see D. Louani, [5]). 

Our second aim is to provide pointwise MDP for the averaged stochastic 
approximation algorithm (|1.2p . In this case, we consider more general weight 
sequence defined as g„ = 7 (n) for all n, where 7 is a regularly function with 
exponent (— q), q < min{l — 2a, (1 + a) /2}. 

For any positive sequence (w„) satisfying 

lim Vn — 00, lim — ^ — and lim w„/i„ = (1-4) 

and general bandwidths (/i„), we prove that the sequence 

Vn (Jn {x) - r{x)) 

satisfies a LDP of speed (nhn/v^) and good rate function Ja q x :R^>-M. defined 

by 

l + a-2q fix) e 

■'"^^" {1-qf Var[Y\X^x]J^KHz)dz2- ^^"'^ 

Let us point out that when the weight (qn) is chosen to be a regularly varying 
function with exponent (—a) (e.g. (qn) = (hn)), which is the case when the 
weight (g„) minimizes the asymptotic variance of r„ (see A. Mokkadem et al., 
[9]), the factor (1 + a — 2q) / {1 — q) which is present in (|1.5p can be reduced to 
1/(1 — a), and then we can write 

J ... _ _J: m t^ „ .^ 

"'"^^ {l-a)Var[Y\X^x]^^K^z)dz2- ^ ' 
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Moreover, D. Louani ;5j establish the moderate deviations behaviour for the 
Nadaraya- Watson ([6], [M]) estimator defined as 

I otherwise, 

where 

™n [x) = —— V i^jiiT ( ^ — - ) and ^ [x) = — — V A" 

They prove that, for any positive sequence (w„) satisfying (|1.4p . the sequence 
"^n (?Vi (a;) — r (x)) satisfies a LDP with speed (nhn/v^^) and good rate function 
jL : R ^ M defined by 

■^^ ^*^ " 1/ar [r|X = x] 4 X2 (z) dz^- ^^-^^ 

Recently, A. Mokkadem et al [5] establish the moderate deviations behaviour 
for the semi-recursive version of the Nadaraya- Watson estimator defined as 

where 

They prove that, for any positive sequence (w„) satisfying (|1.4p . the sequence 
Wn (^n (a;) — ?" (a;)) satisfies a LDP with speed (nhn/vj^) and good rate function 
Ja,x : M ^. R defined by 

Then, it follows from (|1.6p . (jl.Sp and ()1.10p . that the rate function obtained in 
the MDP of r„ defined with a weight ((/„) minimizing the asymptotic variance 
of Tn (e.g. (g„) = (/in)) is larger than the one obtained for the Nadaraya- 
Watson kernel estimator (|1.7p and than the one obtained for the semi-recursive 
kernel estimator (jl.91) : this means that the averaged stochastic approximation 
algorithm rn{x) defined with a weight (^n), which is chosen to be a regularly 
varying function with exponent (—a) (e.g. [qn) = (hn)) is more concentrated 
around r{x) than the two others estimators (Nadaraya- Watson ()1.7p and semi- 
recursive (11.91) ). 






otherwise, 


x~XA 


nnd f (t) ^ \' ^ K ^ ^' 


h ) 


ana Jn[x) / , , ^ u 
n^ hi \ hi 
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2. Assumptions and main results 

Let us first define tlie class of positive sequences that will be used in the 
statement of our assumptions. 

Definition 2.1. Let 7 e K and (««)„>! be a nonrandom positive sequence. We 
say that (w„) G GS (7) if 



lim n 



1 _ ^"-1 



= 7. (2.1) 



Condition (|2.1[) was introduced by J. Galambos and E. Seneta [3] to define 
regularly varying sequences (see also R. Bojanic and E. Seneta [T]); it was used in 
A. Mokkadem and M. Pelletier [^ in the context of stochastic approximation al- 
gorithms. Typical sequences in QS (7) are, for 6 G M, 71^ (logn) , n^ (log log n) , 
and so on. 

Let g{s,t) denote the density of the couple {X,Y) (in particular f (x) — 
Jjgg{x, t) dt) , and set a {x) = r (x) f (x) . 

2.1. Pointwise LDP for the averaged stochastic approximation algo- 
rithm ()1.2p . To establish pointwise LDP for r„, we need the following as- 
sumptions. 

(LI) ii' : R — > R is a nonnegative, continuous, bounded function satisfying 
/ju K (z) dz — 1, /jj zK (z) dz — Q and /j^ z'^K (z) dz < 00. 

(L2) i) (7„) := c;5(-q;) with a e]|,l]; lim„^oo"7« (ln(X;fe=i7fe))~ =00. 
a) (hn) = {cn^°-) with a G ]1 - a, (4a - 3) /2[ and c> 0. 
Hi) (g„) = {c'n^'^) with q < min {1 - 2a, (1 + a) /2} and c' > 0. 

(L3) i) g {s,t) is two times continuously differentiable with respect to s. 

ii) For q G {0, 1, 2}, s t-^ J^ fg (s, t) dt is a bounded function continuous 

ai s — X. 

For (7 G [2,3], s I— > J-g^\t\'' g {s,t) dt IS a. bounded function. 

m) For g G {0, 1}, /jj |t|'' |f(x,i) dt < 00, a.nd s ^ J^f^ {s,t) dt is 
a bounded function continuous at s ~ x. 

(L4) For any u G R, t — > /^ exp (uy) g (i, y) dy is continuous at x and bounded. 

The proof of the following comment is given in A. Mokkadem et al. [8]. 
Comment. Notice that (i4) implies that \/m > 0,Vp > 

the function 1 1-^ \y\™ exp{p\y\) g {t,y) dy is bounded. (2.2) 
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Before stating our results, we set 

S+ = {xeR;K (x) > 0} and S-^{xeR;K (x) < 0} 
and for fixed a: G E 

T+ = {y e M; y - r (x) > 0} and r_ = {y G R; y - r {x) < 0} 
Moreover, we set 

0+ = (S'+nr+)u(S'_nT_) and o_ = (S'+nr_)u(5'_nT+) 

The following proposition gives the properties of the functions ^pa,q,x and 
la.q.x', in particular, the behaviour of the rate function Ia,q,x- 

Proposition 2.1 (Properties oi ipa.q.x and Ia,q,x)- 

Let A be the Lebesgue measure on M and let Assumptions (LI) and {LA) hold. 

(i) ipa,q,x is strictly convex, twice continuously differentiable on M, and La^q^x 

is a good rate function on R. 
(ii) Lf X{0-) = 0, Ia,q,x it) = +00, when i < 0, and 

_ I {l-q)l{l-a)\{S+)J{x) if \iS+nT+)>0 
"'"'^^ ' " I {l-q)/{l-a)X{S^)f{x) if A(5_nT_)>0 



Ia,q,x is strictly convex on M and continuous on ]0,+oo[, and for any 
t > 

Ia,q,x (t) = t (^;,^,)~' (t) - V'a,,,. {{^a,q,x)'' W) , (2-3) 



(Hi) Lf X{0-) > 0, then La^q^x is finite and strictly convex on R and 112. 3\} 
holds for any t € M. 

We can now state the LDP for the averaged stochastic approximation algo- 
rithm Ol. 



Theorem 2.2 (Pointwise LDP for the averaged stochastic approximation algo- 
rithm [TS]). 

Let Assumptions (LI) — (L4) hold. Then, the sequence (r„ (x) — r (x)) satisfies 
a LDP with speed (nhn) and rate function defined as follows: 

Ia,q,x it) = t (^;,^,)"' {t) - ^a,q,x (K,,,.)"' W) , 

where if'a,q,x is defined in il.3\) . 
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2.2. Pointwise MDP for the averaged stochastic approximation algo- 
rithm (II. 2p . Let (vn) be a positive sequence; we assume that 

(Ml) iiT : M ^ M is a nonnegative, continuous, bounded function satisfying 
/jj K (z) dz — 1, /jj zK (z) dz = and /^ z^K {z) dz < oo. 

(M2) i) (7„) = t;5(-a) with a €]|,1]; lim„^oon7« (ln(X;fc=i7fe))~ =00. 
a) (/i„) = gs (-a) with a e ]1 - a, (4a - 3) /2[. 
jzi) ((7„) = QS (—(7) with g < min {1 — 2a, (1 + a) /2}. 

(M3) i) 5 (s,i) is two times continuously difFerentiable with respect to s. 

a) For q G {0, 1, 2}, s i— > L i'^g (s, t) dt is a bounded function continuous 

a,i s — X. 

For 5 G [2,3], s H' /g |t|'' g (s, t) dt is a bounded function. 

m) For g G {0,1}, J^\t\'' || (x,t) rfi < 00, and s ^ J^fi^ [s,t)dt is 
a bounded function continuous at s ~ x. 

(M4) For any u e R, t — > J^ exp (uy) g (i, y) dy is continuous at x and bounded. 

2 
(M5) i) hm„^oo Wn = 00 and hm„^oo j^ = 0. 

iz) hm„^oo Vnh\ = 

The foUowing Theorem gives the pointwise MDP for the averaged stochastic 
approximation algorithm ()1.2p . 



Theorem 2.3 (Pointwise MDP for the averaged stochastic approximation al- 
gorithm pT^ ). 

Let Assumptions {Ml) — (M5) /loZd. Then, the sequence (w„ (r„ (x) — r (a;))) 
satisfies a MDP with speed (nhn/v'^) and good rate function Ja q x defined in 



3. Proofs 

From now on, we set no > 3 such that Vfc > no, 7fc < (2|l/||oo)~ and 
7/c/i^^||iir|joo < 1- Moreover, we introduce the following notations: 



Zn{x) - h~^K( 



Wn{x) = h-^Y,,K 



X ~ Xn 
hn 

X- Xn 
hn 



Vn{x) = {Y^-r{x))K[^-^], (3.1) 
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As explained in the introduction, we note that the stochastic approximation 
algorithm ()l.ip can be rewritten as: 

rnix) = (1 -7„Z„(a;))r„_i(a;) +7„W„ (x) 

= (1 - Inf (x)) r„_i(x) + 7„ (/ (x) - Zn (x)) r„_i (x) + 7„W„ (x) . 

To establish the asymptotic behaviour of (r„) and (r„), we introduce the auxil- 
iary stochastic approximation algorithm defined by setting p„ (x) = r (x) for all 
n < no - 2, p„o-i (x) = rno-i (x), and, for n > no, 

Pn{x) = {l--fnf (x)) pn-l{x) +Jnifix) - Zn{x))r{x) +JnWnix) . 

It follows that, for n > tiq, 

Pn (x) - Pn-i (x) = -7„/(a;) [/9„_i(a;) -r(x)] +7„ [W"„ (x) - r (2;)Z„ (x)] , 
= -7n/ (a;) [p«-i(a;) ~ r (x)] + 7„/i;^^ry„ (x) , 

and thus 



Pn-i(a;) -r(x) = 
Then, we can write that 

p„ (x) - r (x) 



/(^) 



Vn (x) 



1 



7«/ (2^: 



[p„(x) -/9„_l(x)] 



with 



T'n (x) = 

i?i°)(x) = 



1 " 

^^i Y] Ik [Pk {x) - r (a 

Ek=i Ik fr[ 



1 " 

1 " 

^F^^n Y] [Pk+l{x) - Pk{x)] 

l^k=no-l'ik ^ 7fc+l 



(3.2) 



Moreover, it was showen in A. Mokkadem et al ^Hj, that under the assumptions 
(Ml) - (Af3), we have 



i?i")(x)| = o ( Vn-i/i^i + /i;2) 
then, it follows from p.2p and p.Sp that 

1 1 " 



a.s., 



(3.3) 



p„ (x) - E [p„ (x)] = 



/ {x) ELi 9fc 



5] gfe/i^i(r;,(x)-E[r;fe(x)]). 



k—7iQ —1 
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Let {"^n), (Bn) and (A„) be the sequences defined as 
1 1 " 

Bn {x) = E [p„ (x)] ~r{x), 

An {x) = r„ (a;) - p„ (x) . 
We have: 

r„ (x) - r (x) = *„ (x) + B„ (x) + A„ (x) . (3.4) 

Theorems 12.21 and 12.31 are consequences of (|3.4I) and the following propositions. 

Proposition 3.1 (Pointwise LDP and MDP for (*„)). 

(1) Under the assumptions (LI) — {LA), the sequence p„ (x) — E [p„ (x)] sat- 
isfies a LDP with speed {nhn) and rate Junction la.q.x- 

(2) Under the assumptions (Af 1) — [Mb), the sequence (w„^n (x)) satisfies 
a LDP with speed [nhn/v^ and rate function Ja,q,x- 

Proposition 3.2 (Convergence rate of {Bn)). 
Let Assumptions (A/1) — (Af3) hold. Then 

i?„ (x) - O (hi) . 

The proof of the following proposition is given in A. Mokkadem et al. [S]. 

Proposition 3.3 (Convergence rate of (A„)). 
Let Assumptions (Ml) — (Af3) hold. Then 

A„ (x) = o (hi + -i= 



Set X G K; since the assumptions of Theorems l2.2| guarantee that lim„_^oo Bn (x) 
and lim„_>oo A„ (x) = Theorem 12.21 is a straightforward consequence of 
the application of Proposition 13.11 Moreover, under the assumptions of Theo- 
rem 12. 3[ we have by application of Propostion 13.21 lim„_j.oo VnBn (x) = and 
lim„_i.oo v„A„ (x) — 0; Theorem 12.31 thus straightfuUy follows from the applica- 
tion of Part 2 of Proposition 13. II 

We now state a preliminary lemma, which will be used in the proof of Proposi- 
tion [3lll For any m g R, set 

K.x{u) = ^logE[exp(un/i„*„(x))], 
nhn 

^x (w) = '4^a,q,x (u) , 

2 l-|-a-2g / (x) Jr 
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Lemma 3.1 (Pointwise convergence of h.n,x)- 
For allueR 

lim A„_a; (u) = Ax (u) 



whe 



Aa:iu)^s .M 



A^ (u) when Vn = 1, {LI) - {LA) hold 



Kf {u) when v„ -^ oo, (Ml) - (A/4) hold 



Our proofs are now organized as follows: Lemma |3. II is proved in Section [3. 11 
Proposition 13. II in Section [3^ and Proposition 13.21 in Section [331 



3.1. Proof of Lemma I3.1i 

Proof. 

Set li G R, Un = u/vn and a„ — nhn- We have: 



An^x {u) 



■logE[exp(w„a„*„(x))] 



■log 



exp 



/ {^) ELi Ik 



Y, ^ir^,ix)-E[rj,{x)]) 



exp u- 



k—no—l 

Q« gfc ??fc (a^) 



— > log 

k—nti — 1 

n 



By Taylor expansion, there exists Ck,n between 1 and 
that 



exp I Un f^^ y(^) 



such 



logE 



exp Un^^:^ 



a-n Qk r/k {x) 



E 



exp Un 



^'^k,n 



a« gfc Vk jx) 

an Qk Vk {x) 
'J2l=iQkhk f{x) 



E 



exp u. 
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and An x can be rewriten as 



2 

A„.4u) = ^ V E 

k—riQ — 1 
1 2 

1 wi 



exp u 






ELi <?fe ^fc / (x) 

a» gfc T?fc jx) 

' ELi 9/c /ife / (x) 



exp u- 



/(2;)Efc=ift,.jt'_i^ 



E fE[^fc(^)]' 



(3.5) 



Now, let us recall that, if (6„) e GS {—b*) with 6* < 1, then we have, for any 
fixed ko > 1, 



lini 



nfcr, 



ELfco ^fc 



1-5* 



and 



sup — < oo. 

k<n Ok 

Moreover, since (qkh^^) G GS (— (g — a)), it follows from p.6p that 

an % 



(3.6) 



(3.7) 



^ V~>" J. 

Z^fc=i Qk rik 



and from p.7p that 



qk 



t^n ^fe gn 



O (1) when «„ = 1 
o ( 1 ) when -y„ — ?► oo 



' I]fc=i qk hk 
and thus, in the both cases, there exists c > such that 

o-n qk 



Efc=i 9fe hk 



< c. 



First case: u„ -^ cx). 

A Taylor's expansion implies the existence of c'f, „ between and u„ .r^„°" — ^ ttT 

such that 



(3.8) 

o^i Qk Vk(x) 



E 



exp M. 



an qk Vk jx) 
ELi Ik hk f (x) 
Un a„ qk 



- 1 



/ (x) ELi 9fe /ifc 

-1- / ^n 0!-n 



^ r / M 1 f U. 



gfc 



2 \/(2;) ELi^fe hk 



E [r;2 (x)] 



gfe 



6 \f{x) J2k=i1khk 



4 {x) e"""^ 
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Therefore, 



2(/(^)f (ELi*) .ir-1^' 



1 U^Mn g^ y^ g^^ 



1^2 

2 



- E ^ 



E 



J2 ||E[,7l(^)exp(4,„)] 

On gfc ??fe (a;) ~ 



exp u. 



Efe=i ft ^fe / (2^) 



Let us note that under the assumption (M3), we have 

E [ri (x)] = hkVar [Y\X = x] / (x) / K^ (z) dz[l + o (1)] 



Then, it foUows that 
,2 



A„,a; (u) 



U 



(Z^A:=1 ft) fc=n(,-l 



_i:i^^^^m^//'"'^'--°<'> 



(3.9) 



with 



^i'l(«) = 



^EW 



1 m2u„ 



^(/(^)) (Efe=ift) fe=„o-ifc 



li;2 
_ ^ 
2 a„ 



" 1 



exp u 



fc=no — 1 "fe." 

Let us first show that 

hm i?WH 

n— >cxD ' 

In view of (M4) and (13.81) . we have 



fln gfc ^7/= (a^) 

'ELift 'ife /(a;) 



E 



ft (2;)^ exp (4 



|3 7^3, 



< ^fe / \y-r{x)\K-^{z)expl——\y-r{x)\\K{z)\]g{x-zhk,y)dydz 

< 4/ife / exp(^|r(x)|||K||^')(/|y|3expf-f^|y|||X||^).g(a;-z/ifc,y)dy 



/(^) 



/W 



+ |r(x)r / exp( — -y lyllli^ll^ )5(x-z/ifc,y)dyJ.A'3(z)dz 
0{hk). 



(3.10) 
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Hence, it follows from (|3.10|) and (|3.6p . that 



u'^Un 






= o 

= o 
= o 



1 2 "'6 



rig„ 



1 

' n 

k=i9k 
1 



Z^fe=no-l yfc"fc 



ng; 






which ensures that lim„_ 
Let us now prove that 



R^ni {U) 



= 0. 



lim 

n— >oo 



R'n}Au) 



Noting that, under the assumption {Mi) we have 

¥.{Wk{x)) = a{x) + hl f y^{x,y)dy f z'K{z)dz[l + o{l)], 
E(Zfe(x)) = f{x) + ^hll^^ix,y)dyl^z^Kiz)dz[l + o{l)]. 

Then, it follows from p.ip that 

E[r,fc(x)] - /ifc[IE(W^fe(a;))-r(a;)K(Zfc(a;))] 



o/^^ 



J y^{^^y)dy-T{^) j y^(^^v)dy 

X / z^K{z)dz[l + o{\)] 

Jr 

= hlm^'Hx) fix) [1 + 0(1)], 



(3.11) 



where, 

m ' [x) 



V{x) 



f d'^g f d'^g 



z^K{z)dz. 
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Moreover, in view of p.6p and p. lip , we have 



,,2 " 



< Y — 

"" fe=no-l ^fc.« 
2 " 

an ^ 

fc— no — 1 



E 



, fln g/c % (x) \ 
CXp Wn;^:^7J 7 TTT ^ ^ 



< 



E 



exp 



/ On gfc ??fc(x) \ "|\ 

v ELi9fe^'fc /(2;) / J/ 



''" fcJ^-1 V L Efe=i 9fe hk f{x)\J 

= 777^"" E U^^lE[^fe(^)] (1 + 0(1)) 

^ { l^k=no~\ Ik^k \ 

= O a„-— 2 ^ 

V [Ek=l1k) J 

= o {hi) 



which goes to as n — ?> oo. Which proves that hni„^oo Rn,x (u) 

we obtain from p.9p and p.6p . hni„_>oo K^^x {u) — A*^ (u). 
Second case: (ti„) = 1. 
It foUows from (13.51) that 



(3.12) 



0. Then, 



A„,a; (m) 






k^riQ — 1 
n 



, a„ gfc r]k (x) 

exp W=:^^j -— - 

Lfc=i gfc hk i [x) 



2a„ ^^ c? V 

" fc=no-l k.n \ 
U 1 



exp u 



On gfc ??fc [x) \ _ ^ 



ELi gfc ^fe / (2;) 



/ (2;) ELi gfc . ^^ >/= 



E f:nrik{x)] 

k—riQ —1 



k—no — l 

^9{x,y)dzdy 

-i?5(w)-i?w (") + 4'L(^) 



'^'^P 1 TT^^^ — 'T' ^y~^ (^)) K{z)] -1 

(3.13) 
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with 



R'^ui iu) 



R^niiu) - 



2^ ^ — r 



exp u 



a-n gfc Vk jx) 






^n.x \r- 






exp 



gfc 



/c^riQ — 1 

x[g{x- zhk,y) - g {x, y)] dzdy 



f (x) ELi (Ik hk 



{y-r{x))K{z)]-l 



It follows from p.l2p . that hm. 

Moreover, in view of p.6p and p.lip . we have 



^71— >oo ^n,x [Uj 



= 0. 



i?(4) 



o 



= o 



' 1 V .^^ 

En 7 9 



which goes to as n — )■ oo. 
Let us now prove that 



lim 



R^^l in) 



= 0. 



Set M > and e > 0; we then have 



^n,x \^ 



"" fc=„o-l h\A<M}> 



exp 



(?fc 



/ (2:) ELi gfc 'I'c 



(2/-r(a;))i^(z) -1 



X [5 {x - zhk,y)- g {x, y)] dzdy 



-- j: hJ 

an ,^^ ^ J{\, 



exp 



2|>M}xl 

x[g{x- zhk, y) - g (x, y)] dzdy 
I + 11. 



U Qj, 



qk 



f (x) ELl Ik hk 



{y-r{x))K{z)\-l 
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Using p.8|) . and since for any i G M, |e* — 1| < |t| e'*', we have 

n 

Qk 



\II\ < \U\ J2 X-n , 



\y-rix)\\Kiz)\ 



X exp c 



/(^) 



y-r{x)\ \K{z)\ ] \g {x ~ zhk,y) - g {x,y)\dzdy 






Kiz)\ 



\y-r{x)\e^p { Cy-^ |2;-r(a;)| \K {z)\ ] g{x~ zhk,y)dy 



\u\ Y. ^^^^ \Kiz)\ 



dz 



u 



|2/-r(a:)|exp ( c—j--\y-r{x)\\K{z)\ ]g{x,y)dy 



dz 



< A / \K{z)\dz, 

J{\z\>M} 

where A is a constant; this last inequahty fohows from ()2.2p and from the fact 

that K is bounded. 

Now, since K is integrable, we can choose M such that 

Now, for /, we write 



I = ± ± hj 

«» fe=„„-l ^{|-I<M}> 



exp 



a-n qk 



{y-r{x))K{z) 



fc— no — 

X [g (a; - zft,fe, y)- g (x, y)] dzdy 



[g {x - zhk, y)- g [x, y)] dzdy 



In view of (M4), (|2.2p . p. 61) . the dominated convergence theorem ensure that 
both integrals converge to 0. We deduce that for n large enough, 



1 " 

/c— no — 1 



{|2:|<M}XI 



which ensures that lim^ 



R^nl (U) 



0. 



Then, it follows from p.l3p . and (I3.6P and from some analysis considerations 
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a—q 

U 



{y-r{x))K{z)\-l 



' (x, y) dzdy 



.nj fix) 
= (l-q) f ■s-'' f exp (us'^-'^K [z) ^fM") _ ^ g (^, y) dsdzdy 

J[0,l]xR2 V V fix) J J 

and thus Lemnia 1 is proved. D 

3.2. Proof of Proposition [3. li 

Proof. 

To prove Proposition l3.11 we apply Proposition l2.11 Leninia [3.1l and the fohowing 

result (see A. A. Puhalskii, [TU]). 

Lemma 3.2. Let (Zn) be a sequence of real random variables, {vn) a positive 
sequence satisfying lini„_i.oo Vn — +oo, and suppose that there exists some convex 
non-negative function T defined on M such that 

\fu e M, hm — logE [exp (ui^„Z„)] ^T{u). 

// the Legendre function T* ofV is a strictly convex function, then the sequence 
(Zn) satisfies a LDP of speed (vn) o,nd good rate fonction T* . 

In our framework, when «„ = 1, we take Z„ = p„ {x) — E (]o„ (a;)), Vn ~ nhn 
with hn = cn^°- where c > and a e ]1 - a, (4a - 3) /2[ (with a e]|,l]), 
and the weight (q„) = {c'n~'^) with c' > and q < min{l — 2a, (1 + a) /2}, 
and r — A^. In this case, the Legendre transform of F = A^ is the rate 
function Ia,q,x (t) which is strictly convex by Proposition 12. II Otherwise, when, 
"!;„,—>■ oo, we take Z„ — w„ (p„ (x) — E [p„ (a;)]), Vn = nhn/v"^ and F — A*^; F* is 



then the quadratic rate function Ja,q^x defined in (II. 5p and thus Proposition 13. II 
follows. D 

3.3. Proof of Proposition [3T2I 
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Proof. 

It follows from ([321), ^M, dSH) and ([3lT|) . that 



E[p^ix)]~rix) 



1 



1 



E [r„ (x)] 
1 



1 ELnn-i'Jfe/ji 



E fE[ryfc(a:)] 



/ (^) ELi 9fe 



m(2)(:r)/(x)[l + o(l)] 



1-g 



" 1 - 9 - 2a 



to(2) (x) [1 + o(1)] 



n 



3.4. Proof of Proposition [2.11 



Proof. 



Since |e* - 1| < |i| el*l V< G R, it follows from (g^l) and (LI), that 



\i^a,q,x{u)\ < {I- q) I S 

'[0,l]xIE 



< (1-g) 



exp us°--'^K (z) 



{y-r{x)) 



X exp |u 



s-'|y-r(x)||if(z)| 
g {x,y) dsdzdy 



g (x, y) dsdzdy 



< 



/(^) 



< oo 



f (x) 7[0,l]xR2 

\y-r{x)\\K{z)\exp(^\u\^y^^\\K\\^]g{x,y)dzdy 

\y~r{x)\ 



\Kiz)\dz / |y-r(x)|exp |u| 



fi^) 



\\K\\oo I g{x,y)dy 
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which ensures the existence of '4'a,q,x- It is straightforward to check that 
''pa,q,x is twice differentiable, with 

<,q,xW) - (1-?)/ s-'^Kiz)^!-^ 

X exp (us''-''K (z) (^~^(^)) j g (2,^ y) dsdzdy 

<,,.(-) = (1-9)/ ^"^-^'(i^Wf f^^TTT^)' 

i[0,l]xR2 V /W / 

xexp {us^-'^K [z) ^^ ~ ^ V I g (a;, ;/) dsdzdy. 
\ J[x) J 

Since i/'a.q.a; (w) > Vu G R, V'a.g.i i^ increasing on R, and ■(/'a,q,a: is 
strictly convex on JR. It follows that its Cramer transform Ia,q,x is a 
good rate function on R (see A. Dembo and O. Zeitouni [2]) and (i) of 
Proposition 12 . 1 1 is proved. 
• Let us now assume that A (0-) — 0. We then have 

lim i'Lq.x (") = and lim ip'^ ^^ (u) = +oo 
so that the range of ipa,q,x ^^ 1^' -^oo[. Moreover 

,. ., .„^ _ ( -il-q)/{l-a)XiS+)fix) if A(5+nT+)>0 

mn^Va,q,x[u) \ -{l-q)/{l-a)X{S-)f{x) if A(5_nr_)>0 



u—^ — oo 



(which can be — oo). This imphes in particular that 

_ ( il-q)/il-a)XiS+)f{x) if A(5+nT+)>0 
"'«'"^^ \ (l-g)/(l-a)A(5_)/(a;) if A(5_nr_)>0 

Now, when t < 0, linitj_>_oo (ui — ipa,q,x {u)) = +00 and la.q.x {t) — +00. 
Since ipa,q,x is increasing with range ]0, +00 [, when t > 0, sup„ {ut — ipa,q,x (")) 
is reached for uq (t) such that 'ipa,q,x {uo (t)) — t, i.e. for uq {t) = 
{i^'a,q.xy^ (t); this prove ^^. (Note that, since Va.q.a; (0 > 0, the func- 
tion t !—>■ Uq (t) is differentiable on ]0, +oo[). Now, differentiating p.3p . 
we have 

iLq.xit) = wo(t)+tu;,(t)-C,(uo(t))uo(i) 
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Since [%p'^ ^j is an increasing function on ]0, +00 [, it follows that Ia,q,x 
is strictly convex on ]0,+oo[ (and differentiable) . Thus (ii) is proved. 

• We Assume that A (0-) > 0. In this case, ip'a „ x can be rewritten as 
<,,.(«) = (1-9)/ .-^X(z)fcf^ 

X exp (us'^-'^K (z) i^_lMl ) g (x, y) dsdzdy 

V / yx) J ■ 

+ (1-,)/ „-,A-W<i^ 

X exp (us'^-'iK (z) fc ^iMl ) g (x, y) dsdzdy 
\ f i^) J 

and we have 

lim '^'a,q,x (") = -00 and liin tP' (u) = +00 

so that the range of tpa.q.x is ^ in this case. The proof of (Hi) follows 
the same lines as previously, except that, in the present case, (i/"^ „ ^) 
is defined on M, and not only on ]0, +00 [. 

n 
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